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SIMULATION-BASED  MISSION  REHEARSAL 
AS  A  HUMAN  ACTIVITY  SYSTEM 


INTRODUCTION 


Simulation-based  mission  rehearsal  (MR)  became  a 
reality  for  the  Air  Force  in  1990  when  the  MH-53J 
Weapon  System  Trainer  and  Mission  Rehearsal 
System  (WST/MRS)  was  delivered  to  die  542nd  Crew 
Training  Wing  (now  the  58th  Special  Operations 
Wing,  or  58  SOA^  at  Kirtland  AFB,  NM  by  a  group 
widiin  General  Electric  (who  have  since  become  part 
of  Lockheed-Martin).  It  was  the  ability  of  this 
government-owned,  contractor-operated  system  to  use 
’’real  world”  photo-texture  derived  from  imagery, 
combined  with  Defense  Mapping  Agency  (DMA) 
terrain  and  cultural  feature  data,  that  provided  the 
Special  Operations  Forces  (SOF)  and  Combat  Search 
and  Rescue  (SAR)  forces  their  first  MR  capability 
(Reed,  1993).  This  use  of  simulation  added  a  level  of 
realism  to  rehearsal  that  is  not  duplicated  by  other 
media.  Planners  have  access  to  three-dimensional, 
computer-generated  imagery  based  on  Defense 
Mapping  Agency  data  enhanced  with  cultural  features 
and  other  details  from  a  variety  of  sources,  and  mission 
participants  can  interact  with  other  players  in 
networked  simulators.  Demonstrated  benefits  include 
man-in-the-loop  validation  of  mission  plans  leading  to 
improved  mission  plans  and  tactics,  heightened  crew 
confidence  and  preparation,  and  a  clearer 
understanding  of  mission  dynamics  and  risks.  (See, 
for  example,  Nullmeyer,  Bruce,  Conquest,  and  Reed, 
1992). 

In  numerous  rehearsals  for  training  exercises,  we 
have  seen  that  MR  is,  first  and  foremost,  a 
cooperative  venture  among  people  to  get 
crewmembers  maximally  prepared  for  their  mission, 
and  it  is  these  varied  people  who  will  determine  the 
effectiveness  of  advanced  simulation  technology  for 
this  purpose.  Integrating  simulation-based  rehearsal 
capabilities  with  the  existing  functionally. 


geographically,  and  organizationally  diverse  SOF  and 
SAR  mission  preparation  process  is  clearly  a 
challenge.  A  key  element  for  meeting  this  challenge 
was  realizing  that  simulation-based  rehearsal 
technology  and  its  uses  by  people  (both  Air  Force  and 
contractor  personnel)  must  be  viewed  as  an  integrated 
system  that  we  will  refer  to  as  a  human  activity  system. 
This  point  of  view  provides  a  conceptual  structure  to 
organize,  understand,  and  facilitate  the  diversified 
activities  of  tiie  many  participants  involved. 

Banathy  (1992)  asserted  that  no  single  model  can 
truly  represent  a  complex  human  activity  system.  He 
was  ad^essing  educational  systems.  We  believe  his 
concepts  are  equally  useful  for  understanding 
simulation-based  rehearsal.  He  proposed  three 
models  to  capture  the  essence  of  such  systems:  a 
systems-environment  model,  a  functions/structure 
model,  and  a  process  model.  The  systems- 
environment  model  describes  the  context  in  which 
the  system  operates  and  provides  the  “big  picture” 
for  the  remaining  models.  The  functions/structure 
model  describes  the  components  or  parts,  their 
functions,  how  they  are  organized,  and  how  these 
elements  are  integrated  into  a  structured  system. 
Finally,  the  process  model  provides  a  “motion 
picture”  image  of  the  system,  addressing  the  receipt 
and  transformation  of  inputs  and  other  dynamic 
operations  associated  with  the  system,  including  a 
feedback  loop  to  support  system  management, 
adjustment,  and  change.  Each  model  portrays  certain 
critical  characteristics  that  must  be  overlaid  upon 
each  other  to  paint  a  comprehensive  picture  of  the 
system  as  a  whole.  We  will  employ  these  “lenses” 
to  illustrate  the  importance  of  people  in  effective 
simulation-based  MR,  using  the  58  SOW  MRS  as  the 
prototype. 


A  HUMAN  ACTIVITY  SYSTEM  MODEL  OF  SIMULATION-BASED  MR 


The  Environment  -  Crisis  Action  Planning 

Because  simulation-based  MRSs  are  designed  to 
support  short-suspense  missions,  we  chose  to  address 
the  MRS  within  context  of  crisis  action  planning. 
Crisis  action  planning  is  a  coordinated  sequence  of 


events  involving  people,  procedures, 
communications,  and  support  systems  that  leads  to  a 
military  response  to  a  time-sensitive  situation.  The 
primary  participating  organizations  are  listed  on  the 
left  side  of  Figure  1.  TTie  major  events  and  products 
for  these  organizations  are  depicted  to  the 
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Figure  1:  Overview  of  the  Crisis  Action  Planning  "Environment” 


right  of  each  group.  Arrows  show  a  notional 
sequencing  of  these  elements,  although  requirements 
often  force  deviations  fi*om  this  template.  Crisis 
action  planning  begins  with  the  occurrence  of  an 
event  having  possible  national  security  implications. 
The  theater  commander  reports  the  event  and 
provides  his  assessment  to  the  National  Command 
Authorities  (NCA)  and  the  Chairman  of  the  Joint 
Chiefs  of  Staff  (CJCS).  The  NCA  evaluate  the 
situation,  determine  whether  a  crisis  exists,  consider 
military  and  non-military  options,  and  decide 
whether  to  prepare  for  a  possible  military  course  of 
action  (COA).  The  CJCS  may  issue  a  Warning  Order 
to  the  theater  commander-in-chief  (CINC).  It  defines 
command  relationships,  the  mission,  and  resource 
requirements.  Our  focus  for  now  will  be  on  the 
places  where  wing  participation  is  likely.  Figure  1 
shows  three  areas  of  potential  participation  for 
supporting  commands  (and  their  subordinate  units)  -- 
developing  alternative  courses  of  action,  execution 
planning,  and  mission  execution.  All  three  could 
benefit  from  access  to  an  MRS,  as  will  be  described 
in  subsequent  discussions. 

Figure  2  provides  a  more  detailed  look  at  one  of 
these  three,  execution  planning.  Square  boxes  depict 


the  activities  of  aircrews  who  would  plan  and  execute 
the  mission.  The  wing  establishes  a  planning  cell 
upon  receipt  of  a  mission  tasking  in  the  form  of  a 
warning,  planning,  or  alert  order,  general  planning 
begins  with  an  initial  briefing  to  the  planning  cell  to 
communicate  planning  to  date,  constraints, 
intelligence  data,  and  situation  updates.  The 
planning  cell  then  gathers  additional  information 
such  as  charts  and  imagery,  and  performs  multiple 
analyses  across  various  information  categories  such 
as  weather  patterns,  weight  and  balance  data, 
possible  aerial  refueling  tracks,  high  altitude  and  low 
level  route  options,  objective  area  terrain,  threat 
coverage,  and  order  of  battle  data  to  develop  a 
general  plan  that  includes  possible  landing  zones, 
tactics,  escape  and  evasion  routes,  and 
communications  procedures.  An  interim  assessment 
of  plan  viability  is  performed  at  the  end  of  each 
planning  cycle.  Simulation  technology  has  several 
uses  at  this  early  point  in  planning.  Planners  could 
use  even  an  unenhanced  digital  database  to  analyze 
the  terrain,  assess  the  viability  of  basis  tactics,  and 
study  the  terminal  area  to  determine  the  number  and 
type  of  aircraft  that  can  be  inserted.  Historically, 
these  tasks  were  performed  using  paper  maps,  table 
look-ups,  and  rules  of  thumb  estimates. 
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Figure  2:  An  Overview  of  Wing  and  Squadron  Activities  in  the  Execution  Planning  Phase. 


An  aircrew  briefing  transitions  the  general  plan  to 
squadron  crewmembers  who  will  complete  the 
planning  details.  During  this  handover,  the  crews  are 
briefed  on  the  mission  tasking,  intelligence 
information,  predicted  weather,  and  available 
logistics.  During  detailed  planning,  crews  insert 
specific  elements  into  the  general  plan  and  make  final 
decisions  concerning  key  aspects  of  the  mission. 
Activities  are  laborious  and  include  completing  high- 
altitude  and  low-level  routes  -  specifying  waypoints, 
initial  points,  and  other  navigation  data;  finalizing 
tanker  track  and  ground-based  refueling  station 
locations;  selecting  landing  zones  and  holding  areas; 
determining  terminal  area  tactics  and  procedures; 
calculating  weight  &  balance  and  fuel  flow; 
determining  procedures  to  avoid  threats,  populated 
areas,  and  lines  of  communication;  and  completing 
the  communication  matrix,  load  plan,  and  mission 
execution  checklist.  Simulation  can  support  many  of 
these  functions,  such  as  deconflicting  routes  among 
multiple  aircraft,  determining  suitable  sites  for 
landing  zones,  identifying  methods  for  optimizing 
aircraft  performance,  honing  tactics,  and  coordinating 
practice  on  perishable  tasks  like  threat  avoidance. 

During  evaluation/rehearsal  plan  details  are  fine- 
tuned  and  assessed  under  various  “what-if’ 
contingencies.  Commonly  planned-for  contingencies 
include  the  appearance  of  new  threats,  take-off  or 
landing  with  reduced  engines,  bad  weather,  loss  of 


communications,  loss  of  aircraft,  and  key  equipment 
failure.  Here,  simulation  allows  man-in-the-loop 
validation  of  mission  plans  and  tactics,  which 
heightens  crew  confidence  in  themselves  and  the  plan. 
Based  on  assessments  during  rehearsals  and  crew 
“brief  backs,”  commanders  can  better  estimate  the 
probability  of  mission  success.  In  final  preparation, 
crews  coordinate  with  other  participants;  finalize  the 
mission  execution  checklist;  incorporate  the  latest 
mission,  weather,  and  threat  data;  and  complete 
aircraft  configuration  and  flight  procedures 
immediately  prior  to  executing  the  mission. 

MRS  Components  and  Their  Functional 
Relationships 

Following  the  human  activity  system  point  of  view, 
the  people,  their  functions,  and  their  organization  are 
the  system  of  interest.  Planning  and  rehearsal 
technologies  are  tools  that  support  these  people. 
Figure  3  depicts  the  people  who  are  essential  for 
effective  simulation-based  MR.  Solid  arrows  reflect 
person-to-person  communication,  and  dashed  arrows 
depict  equipment-mediated  information  flow.  It  is 
important  to  note  that  this  model  extends  beyond  the 
local  58  SOW  and  MR  contractor  organizations.  The 
tasking  agency  must  provide  the  geographic 
boundaries  to  be  modeled,  timeline,  available  source 
data,  routes  to  be  flown,  initial  products  desired 
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Figure  3.  Human  Components  of  a  Simulation-Based  Mission  Rehearsal  System 


(hardcopy,  videotape),  and  authorize  various  data 
providers  to  release  mission  information  so  that 
database  development  may  proceed.  The  quality  of 
the  rehearsal  environment  is  critically  dependent  on 
Data  Providers  such  as  planners,  weather  officers, 
and  intelligence  officers. 

5Sth  Operations  Group  Aircrew  Training  System 
Division  (58  QG/QGIB  and  squadron  intelligence 
personnel  provide  tasking  and  mission  information 
to  the  MRS  tactical  analysts  on  the  contractor  team. 
These  analysts  study  the  tasking  along  with  annotated 
maps,  imagery,  and  threat  information  and  begin 
prioritizing  areas  to  receive  high-  and  mid-levels  of 
detailed  database  development.  Clear  lines  of 
communication  between  MRS  personnel  and  the 
supported  organization  are  vital  to  insure  that  the 
latest  and  highest  fidelity  source  imagery  is  available, 
the  most  recent  routes  are  conveyed,  and  the  general 
tactical  plan  is  relayed.  Otherwise,  precious  time  is 
spent  building  databases  for  areas  that  are  not  part  of 
the  operation,  less  accurate  geographic  information  is 
incorporated  into  the  database,  and  fewer  tactically 
relevant  features  are  modeled. 

MRS  database  developers  use  state-of-the-art 
software  and  hardware  to  scan  maps,  warp  imagery, 
extract  elevation  data,  manipulate  contour  lines,  and 
construct  object  models  to  create  a  realistic  database 
of  the  mission  area.  Information  is  exchanged  among 
analysts  and  programmers  using  a  mix  of  verbal, 
written,  and  electronic  means.  The  quality  of  the 


source  data  provided  to  the  developers — be  it  target 
imagery,  hand  held  photos,  or  terrain  information — is 
a  major  determinant  of  the  accuracy  and  realism  of 
the  resulting  digital  database. 

Mission  planners  and  participants  (planning  cell 
staff,  aircrews,  liaisons)  use  the  rehearsal  database  to 
revise,  refine,  and  update  their  mission  plan.  Mission 
information  can  be  delivered  in  a  variety  of  forms, 
such  as  hardcopies  of  enhanced  imagery,  views  of 
partially  modeled  mission  areas  on  work  stations,  and 
completed  versions  of  the  database  in  a  simulator. 
Access  to  the  MRS  by  planners  can  benefit  both 
planning  and  database  development  as  summarized 
on  the  right  hand  side  of  Figure  2.  Early  mission 
planing  decisions  can  help  the  database  generation 
process  by  identifying  the  areas  where  database 
developers  need  to  focus  their  efforts. 

Senior  Decision-makers  (wing,  squadron,  and 
mission  commanders;  executive  and  action  officers; 
and  flight  leaders)  are  clearly  pivotal  participants  in 
the  mission  preparation  process,  and  are  another 
essential  element  of  the  MR  human  activity  system. 
They  can  watch  actual  rehearsals  via  large  repeater 
displays  in  a  training  observation  center  (TOC), 
communicate  directly  with  the  Participating 
Aircrews  over  the  intercom  networks,  and  review 
printouts  of  threat  avoidance,  navigation  accuracy, 
timing,  and  other  indices  via  the  Instructor  Operator 
Station  (lOS).  Using  a  combination  of  viewed 
simulations,  tactical  judgment,  electronic  what-iffing, 
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and  brainstorming,  realistic  estimates  of  mission 
success  can  be  generated  based  on  the  latest  and  best 
available  information.  As  this  information  is 
distilled,  participating  crews  are  "transformed"  into  a 
truly  mission-ready  aircrew  by  rehearsing  the 
planned  mission  in  the  networked  helicopter  and  C- 
130  Aircrew  Training  Devices  (ATDs). 

Figure  4  adds  MRS  technology  to  the  human  activity 
system.  We  represent  equipment  components  and 
information  with  dashed  boxes/arrows  and  the  human 
components  with  solid  boxes  (people)  and  arrows 
(interpersonal  communication).  Interactions  between 
people  and  equipment  are  denoted  by  overlapping 
their  respective  boxes.  The  MRS  receives  input  from 
the  data  providers  in  many  forms,  referred  to 
collectively  as  "all  source  data."  Source  data  is 
voluminous  and  includes  topography, 
meteorological7geodetic,  communications,  command 
and  control,  threat,  and  ownship  performance  data. 
The  MRS  is  configured  to  receive  most  formats  of 
these  data,  including  digital  and  paper  charts  and 


maps,  photography,  imagery,  sketches,  diagrams,  and 
videotape. 

These  data  enter  the  58  SOW  MRS  through  either 
data  analysis  systems  or  electronic  mission 
planning/intelligence  systems.  Tactical  analysts  use 
mission  planning  and  intelligence  systems  to  update 
the  all  source  data  with  real-  or  near-real  time  threat 
location  and  order  of  battle  data.  The  data  analysis 
systems  are  a  constellation  of  computer 
hardware/software  that  preview,  enhance,  modify, 
and  augment  the  all  source  data.  The  Database 
Generation  System  (DBGS)  is  a  vast  array  of 
contractor-operated  hardware  and  software  that 
creates  the  geo-specific  visual  database  of  the 
mission  area.  Intelligence  and  tactical  analysts 
receive  amplifying  information  from  the  data 
providers  and  utilize  the  data  analysis  system  to 
extract  as  much  information  as  possible  from  the  all 
source  data.  Meanwhile,  contractor-supplied 
database  developers  and  terrain  modelers  begin 
constructing  the  digital  database  the  aircrews  will  fly 
through  during  MR. 
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The  major  products  of  the  database  generation 
system  (DBGS)  are  the  multiple  digital  databases 
reflecting  different  environmental  or  sensor 
conditions.  The  visual  out-the-window  (OTW) 
database  can  be  viewed  through  night  vision  goggles 
(NVGs)  and  processed  at  different  moon  illumination 
levels  to  simulate  OTW  night  scenes.  Other 
databases  are  created  to  mimic  the  FLIR,  radar,  and 
EW  environments.  Five  different  "portals"  into  the 
Digital  Databases  are  denoted  by  exiting  arrows.  The 
arrow  at  the  top  leads  to  the  ATDs.  Hardcopy 
printouts  of  any  part  of  the  visual  scene 
(photographs,  maps,  combat  folders,  navigation  logs) 
that  can  be  taken  on  the  aircraft  during  mission 
execution.  A  third  link  is  the  TOC,  in  which  large 
repeater  displays  let  senior  decision-makers  observe 
the  participating  aircrews  fly  through  the  database  in 
the  ATDs.  Other  outputs  include  a  commercial- 
quality 

audio-visual  recording  studio  that  can  record  the 
route  flown  through  the  database;  a  standalone  work 
station  that  lets  aircrews  "joystick"  through  the 
Database  decoupled  from  the  aero-vehicle  model; 
and  a  distributed  interactive  simulation  (DIS)  node. 

Mission  planners  and  participants  can  derive  benefit 
from  the  Databases  by  taking  hardcopy  products 
onboard  the  aircraft,  flying  through  the  database  in 
an  ATD,  and  by  using  the  Audio-Visual  Recording 
System  (AVRS)  to  create  a  video  recording  of  the 
aircraft’s  route  through  the  database  and  record  crew 
comments  during  flight.  Senior  decision  makers  can 
view  the  mission  unfold  by  watching  videotapes  from 
the  AVRS,  the  display  monitors  in  the  TOC,  or 
multi-group  interactions  over  the  DIS  node.  Inputs  to 
mission  execution  come  from  critiques  by  senior 
decision-makers,  the  knowledge  and  experience 
gained  by  mission  planners  and  participating 
aircrews. 

Simulation-Based  Rehearsal  Processes 

The  process  model  describes  the.dynamic  nature  of 
the  system.  While  each  MR  is  unique,  a  core  set  of 
functions  must  be  performed  to  ensure  that  the 
requisite  materials,  information,  support  equipment, 
and  people  have  been  coordinated.  Figure  5  divides 
these  critical  functions  into  three  phases — MR 
preparation,  MR,  and  post-MR.  Although  the 
functions  are  sequenced  temporally  within  each 
phase,  we  have  not  overlaid  an  absolute  timeline 
since  that  depends  on  the  complexity  of  the  operation 
and  whether  an  existing  database  can  be  modified. 


MR  Preparation.  Before  mission  participants  arrive 
at  the  58  SOW  facility,  an  intensive  series  of 
preparation  activities  occur.  An  MR  begins  with  a 
tasking.  A  58  OG/OGU  representative  is  contacted 
to  determine  if  a  particular  operation  can  be 
supported  within  a  given  time  frame.  During  initial 
discussions,  a  58  OG  representative  determines 
which  organizations  will  be  providing  route  and 
targeting  data  as  well  as  transmitting  source  data. 
Internally,  the  OG  representative  will  work  with  the 
designated  MR  Team  Chief  to  line  up  the  requisite 
assets  to  support  an  MR,  such  as  simulator  time, 
AVRS  and  lOS  operators,  data  packs.  Image 
Generators  (IGs)  for  the  proposed  mission,  and 
intersimulator  networking.  To  facilitate  data  analysis 
and  database  development,  the  58  OG  recommends 
what  and  who  the  supported  organizations  should 
send  ahead  and/or  bring  with  them  to  the  training  and 
rehearsal  complex.  The  "what"  entails  identifying 
necessary  maps,  charts,  imagery,  and  intelligence 
data  in  the  region  of  interest.  The  "who"  are  specific 
crewmembers,  intelligence  and  logistics  personnel, 
and  associated  ground  customers  who  should  be  on¬ 
site  during  the  MR. 

Database  development  in  the  MRS  is  an  extremely 
involved  function  that  can  be  organized  into  eight 
steps:  1)  digital  terrain  elevation  data  and  digitized 
source  data  are  combined  to  create  an  underlying 
terrain  grid  of  elevation  values;  2)  cultural  features 
(e.g.,  roads,  rivers,  trees)  are  taken  from  digital 
feature  analysis  data,  photos,  and  images,  and  added 
to  the  terrain;  3)  the  terrain  is  triangulated  into 
polygons  by  positioning  a  given  image  to  an  exact 
latitude/longitude  (warping)  and  elevation 
(orthorectification);  4)  terrain  is  textured  to  appear 
more  realistic  by  applying  texture  models  coupled 
with  recent  real-  world  photos  of  the  mission  area;  5) 
two-  and  three-dimensional  object  models  from  a 
customized  library  and  added  to  the  database;  6)  the 
database  is  compiled,  processed,  and  constructed 
with  concentric  levels  of  resolution.  (Areas  around  a 
target  or  landing  zone  having  the  highest  resolution, 
areas  surrounding  key  landmarks  having  moderate 
resolution,  and  ingress/egress  corridors  having  a 
lower  resolution);  7)  the  database  is  formatted  into 
multiple  layers — OTW/visual,  radar,  FLIR,  EW,  and 
NAVAID — ^and  then  correlated  so  they  can  be 
displayed  from  a  common  viewing  point  and  8)  the 
database  is  processed  through  the  IG  in  which  scene 
objects  are  given  geometrical  perspective,  data  are 
transformed  into  a  display  raster  format,  pixels  are 
given  individualized  color  and  brightness  values. 
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Figure  5.  Functions  and  Products  in  the  Mission  Rehearsal  Process. 


and  stored  in  frame  buffers  for  a  consistent  display 
rate.  During  each  step,  modelers  and  analysts  must 
work  closely  so  that  the  most  important  features  find 
their  way  into  the  database. 

The  visual  database  is  the  most  obvious,  but  by  no 
means,  only  digital  product  for  MR.  Other  products 
are  the  digital  record  of  flight  planning  data,  photos 
and  images  digitized  for  subsequent  manipulation 
(annotation,  mensuration,  warping),  and  scanned 
maps  and  charts.  After  the  database  is  developed, 
government  and  contractor  personnel  pre-fly  the 
database  to  certify  its  spatial  and  content  integrity. 
58  SOW  personnel  can  fly  the  database  at  the 
headings,  altitudes,  and  airspeeds  corresponding  to 
the  mission  plan.  In  Dry  Run  tests,  the  ATDs  will  be 
integrated  with  the  EW  environment  and  wired  for 
full  communication  with  the  TOC. 

MR.  Upon  arriving  at  the  58  SOW,  mission 
participants  will  be  in-briefed  on  MR  objectives  and 
schedule  of  events,  the  contents  of  the  database,  and 
importantly,  what  has  been  omitted  from  the  database 
due  to  visual  system  limits,  time  constraints,  and  lack 
of  available  source  data.  The  "maturity”  of  the 
participant's  plan  will  dictate  the  activities  required 
for  refining  the  mission  plan  and  associated 
database.  Participants  "trial  fly"  mission  segments 
in  an  ATD,  trying  out  different  entry  points  into  the 
area  of  operations,  landing  zones  and  launch  points, 


and  approach  headings  and  profiles.  With  the  initial 
planning  complete,  the  rest  of  the  time  is  spent 
performing  tasks  associated  with  the  primary 
function,  rehearsal.  The  type  of  operation 
(feasibility  assessment,  plan  validation,  plan 
verification,  joint  mission  practice)  will  determine 
the  content  and  extent  of  the  activities  performed 
here.  Typically,  the  TOC  is  "up”  to  monitor  network 
activities,  aircrew  communications,  status  of  the  EW 
environment,  and  coordinate  MR  activities  amongst 
the  various  participants.  Time  permitting, 
participants  will  fly  ingress  routes  into  the  target  area 
multiple  times,  examining  the  impact  of  different 
turning  radii,  approach  angles,  altitudes,  and  multi¬ 
ship  formations.  The  primary  output  of  MR  is  the 
creation  of  a  mission  ready  crew.  Observers  in  the 
TOC,  including  senior  decision  makers,  can  listen  to 
the  audio  on  the  network  and  evaluate  the  integrity  of 
the  visual  database  and  the  performance  of  the  EW 
displays,  and  thus  validate  the  mission  plan  while 
interacting  with  the  rehearsing  crews  in  real  time. 

Post-MR.  Most  MR  operations  will  conclude  with  a 
hot  wash  in  which  mission  participants  and  facility 
personnel  perform  self-critiques  of  the  benefits  and 
negative  aspects  to  the  MR  just  concluded.  These  hot 
washes  have  been  instrumental  in  making 
constructive  improvements  to  the  MR  process,  both 
from  an  efficiency  of  utilization  and  outcome 
effectiveness  standpoint.  58  SOW  personnel  may  use 
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the  AVRS  to  produce  a  videotape  of  the  database 
flown  at  the  headings,  routes,  altitudes,  and  airspeeds 
planned  for  by  the  crew.  With  full  cockpit  audio,  the 
videotape  can  be  edited  to  include  additional  footage 
(e.g.,  360  degree  scans  of  target  areas  fix)m  different 
perspectives,  such  as  ground  level)  to  be  viewed  by 
aircrews  and  decision-makers  at  forward  operating 
locations.  Aircrews  or  the  studio  operator  can  also 


provide  explanatory  voice  overs.  Finally,  mission 
data  and  databases  are  archived  in  the  event  the  plan 
is  subsequently  refined,  re-flown,  or  updated.  Since 
many  operations  take  place  in  the  same  geographic 
region,  the  visual  database  becomes  a  highly  useful 
means  to  store  and  summarize  previous  tactical 
analyses. 


IMPLICATIONS  AND  LESSONS  LEARNED 


The  importance  of  people  in  complex  systems  is 
often  acknowledged.  However,  it  is  not  always  easy 
to  translate  this  concept  into  reality,  as  evidenced  by 
advanced  technologies  being  delivered  with  poor 
user  documentation,  or  with  little  or  no  training  for 
users.  We  beliwe  the  human  activity  system  “lenses” 
provide  useful  templates  to  facilitate  Ais  translation. 
In  this  final  section,  we  will  explore  a  few 
implications  of  these  templates  when  applied  to  the 
58  SOW  MRS. 

First,  the  context  lens  of  crisis  action  planning  shows 
that  the  value  of  MR  can  extend  well  beyond  the 
traditional  view  of  practice  and  validation 
immediately  preceding  mission  execution.  Figure  1 
shows  the  supporting  command  involved  at  three 
points  within  the  crisis  action  planning  process:  (1) 
developing  and  evaluating  alternative  military 
courses  of  action,  (2)  developing  the  detailed 
operations  plans,  and  (3)  executing  the  plan  given 
receipt  of  an  execute  order.  All  three  tasks  can 
benefit  from  access  to  a  digital  database  representing 
the  area  of  operation.  Early  in  the  planning  process, 
alternative  courses  of  action  and  other  planning 
activities  can  be  supported  with  “man-in-the’  loop” 
queries.  Similarly,  simulation  can  provide  a  dynamic 
environment  for  risk  analyses  and  decision  making 
during  mission  execution.  Other  participants  in  crisis 
action  planning  could  also  benefit  from  simulation. 

Second,  the  functions/structure  lens  shows  that  the 
human  system  that  must  be  in  place  for  effective 
simulation-based  MR  clearly  extends  beyond  the 
locally-controlled  MRS.  We  have  seen,  for  example, 
that  the  quality  of  the  MR  database  is  limited  by  the 
quality  of  input  data  from  outside  sources.  MR 
effectiveness  is  also  determined  in  large  part  by  user 
(participants  and  decision  makers)  knowledge  of 
MRS  capabilities,  data  requirements,  capabilities,  and 
utilization  strategies.  Of  some  note,  early  users  of 
simulation-based  MR  uniformly  praised  the 
capabilities  of  the  technology  and  people,  while 


lamenting  the  lack  of  a  strategy  to  incorporate  this 
capability  into  the  mission  preparation  process 
(Nullmeyer,  et.  al.,  1992).  Predictably,  it  has  been 
easier  to  improve  processes  within  the  MRS  than  to 
coordinate  activities  of  the  geographically  and 
organizationally  diverse  team  comprising  the  larger 
mission  preparation  system.^  .  However,  effective 
simulation-based  MR  requires  the  larger  system  to  be 
functioning  as  an  integrated  whole. 

Some  specific  examples  may  help  illustrate  this 
point.  1)  The  MRS  has  a  photo  overlay  capability. 
Annotations  were  traditionally  placed  on  photographs 
to  add  important  and  relevant  information,  but  now, 
these  annotations  add  work  for  database  developers, 
who  must  remove  them  during  database  construction. 
For  a  photo-based  MRS,  it  is  clearly  better  for 
database  developers  to  have  bofti  an  annotated  and  an 
unannotated  photograph.  2)  Experience  has  taught 
that  MR  database  responsiveness  and  quality  is 
enhanced  when  the  user  is  directly  involved  during 
database  development.  Today’s  technology  does  not 
allow  all  things  to  be  modeled  perfectly,  so  database 
developers  attempt  to  find  the  “best  fit”  between 
MRS  capabilities  and  mission  needs.  A  mission 
participant  or  planner  can  provide  valuable  guidance 
to  focus  system  resources  on  the  most  critical  needs. 
This  user  representative  can  also  serve  as  an 
interpreter  for  the  other  rehearsal  participants  to 
describe  where  the  simulation  is  accurate  and  where 
it  is  not.  In  both  examples,  MR  efficiency  and 
effectiveness  are  impacted  by  people  who  are  not 
part  of  the  MRS  per  se. 
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The  functions/structure  model  also  shows  MR 
databases  at  the  center  of  the  MRS.  These  databases 
are  true  “national  assets”  in  the  sense  that  they  can 
easily  contain  more  mission  information,  from 
multiple  sources,  packaged  in  a  more  useable 
structure,  than  is  available  anywhere  else.  With 
proper  packaging  of  database  contents,  many  users 
can  be  served  and  many  uses  can  be  met  with  this 
information.  Using  simulators,  participating  crews 
and  mission  planners  can  conduct  “what  if’ 
exercises,  practice  difficult  maneuvers,  determine 
weapon  system  limits  (e.g.,  maximum  comfortable 
gross  weight  for  a  high  altitude  rotaiy  wing 
operation)  and  conduct  individual  and  collective  risk 
assessments.  Hard  copy  products,  such  as  enhanced 
photographs  and  maps  can  be  generated  in  a  simple 
extension  of  the  database  development  process.  Such 
products  have  been  highly  valued  by  planners  and 
mission  participants.  Videotapes  are  an  additional 
medium  for  exporting  database  information.  This 
medium  provides  a  mission  preview  capability  for 
participants  and  decision  makers  who  do  not  have  the 
luxury  of  physically  going  to  the  rehearsal  system. 
Finally,  the  training  observation  center  and  the 
distributed  interactive  simulation  node  can  connect 
decision  makers  and  mission  participants  to  the 
rehearsal  environment  and  add  a  d3mamic  dimension 
to  the  information  available  upon  which  to  make 
decisions. 

Finally,  Banathy  emphasizes  the  importance  of 
feedback  as  part  of  the  process  model.  An  extensive 
and  diverse  literature  strongly  indicates  that  mere 
repetition  is  not  likely  to  produce  optimal 
performance  (see,  for  example,  Ericsson,  Krampe, 
and  Tesch-Romer,  1993).  Rather,  feedback  is  needed 
to  shape  practice  in  ways  that  encourage  creation  of 
improved  strategies  for  task  accomplishment.  This 
has  relevance  for  both  mission  preparation,  where 
measures  of  mission  effectiveness  are  nWded  to 
provide  the  feedback  needed  to  improve  the  plan  and 
the  performance  of  participants.  Similarly,  feedback 
is  needed  for  the  rehearsal  system,  to  improve  the 
MR  environments,  MRS  elements,  and  related 
processes.  Spiker  and  Nullmeyer  (1995)  developed 
an  initial  set  of  effectiveness  measures  for  the  MRS. 
These  measures  reflect  both  the  context  of  the  larger 
mission  preparation  process,  and  input,  process,  and 
output  components  of  MR.  Initial  input  measures 
include  completeness,  timeliness,  and  relevance  to 
mission  decisions  and  outcomes.  Process  measures 
include  efficiency  and  timeliness  of  database 
development,  and  fidelity  and  relevance  of  the 
resulting  database.  Measures  of  rehearsal  output 


include  the  ability  of  planners  to  create  better  plans, 
crews  having  more  accurate  mental  models  of  the 
operating  environment,  and  decision  makers  having 
better  bases  for  decisions.  Finally,  MR  itself  can 
serve  as  a  valuable  external  feedback  mechanism  for 
training.  Performance  in  the  rehearsal  environment 
can  provide  insights  into  how  well  training  is 
preparing  crews  for  their  operational  missions,  and 
can  identify  gaps  and  oflter  problem  areas.  Lessons 
learned  at  the  58th  SOW  can  help  other  systems 
mature  as  the  “book”  is  still  being  written  on  MR  in 
three  dimensional  visualization  simulation. 
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